Generalized Theory for Deadlock-Free Adaptive Wormhole Routing and its Application to Disha Concurrent

This paper generalizes a theory for deadlock-free adaptive wormhole routing by considering a mixed set of resources: edge and central buffers. This generalized theory is then applied to a concurrent version of Disha deadlock-recovery which relaxes the sequential recovery requirement for simultaneous recovery from deadlocks. The proposed extension to Disha does not necessitate any additional resource cost; rather, it serves to eliminate the requirement of mutual exclusive access to the deadlockfree lane implemented by a Token. With this extension, Disha Concurrent remains applicable to any topology with a Hamiltonian path including k-ary n-cube networks and is also applicable to tree-based networks. 1.0 Introduction: Performance of parallel processor systems hinges on effective utilization of system resources. Communication efficiency of the interconnection network can be enhanced by incorporating wormhole switching and adaptive routing. This potent combination is unfortunately susceptible to deadlocks. Traditional schemes based on avoiding deadlock limit the adaptivity of the routing algorithm [4, 12]. Restricting adaptivity curtails performance and may not allow packets to route around faults. Routing adaptivity can be enhanced by increasing the number of virtual channels [11]. However, additional virtual channels increase the complexity of the router crossbar and the virtual channel controller, leading to longer router clock cycle time and a consequent cost/performance trade-off [5]. Other avoidance schemes [6, 7] designate some virtual channels specifically for the prevention of deadlocks while allowing fully adaptive routing on others. This also leads to reduced utilization of the channels devoted to deadlock avoidance. Deadlock recovery as a viable alternative to avoidance * The research described in this paper was supported in part by an NSF Research Initiation Award, grant ECS-9411587, and by the Spanish CICYT, grant TIC94-0510-C02-01 has only recently begun to gain consideration. Prior research has shown that deadlocks are generally infrequent [2, 9]. Because deadlocks are rare it does not make sense to limit the routing algorithm. This has motivated the development of novel routing strategies based on recovery from deadlocks designed to make the common case fast. Compressionless Routing [9] is one such strategy that detects potential deadlock situations and recovers from them by simply killing the deadlocked packets. Disha† [2] is a more efficient deadlock recovery scheme which is not based on “regressive” abort-and-retry but, rather, on “progressive” redirection of deadlocked packets. Figure 1 shows a flow diagram of the Disha approach. As shown, Disha permits unrestricted routing on all existing virtual channels (i.e., edge buffers), and, thus, results in true fully adaptive routing. If none of these channels are free during this routing cycle, the packet blocks. After several attempts to route the packet, the router may determine that this packet is potentially deadlocked. At this point, a decision is made as to the eligibility of this packet to progressively recover from deadlock using the recovery path. As only one of the packets involved in a deadlock needs to be eliminated from the dependency cycle to break the deadlock, a packet either uses the recovery path or will eventually be able to use one of the normal edge buffers for routing (i.e., deadlock broken by some other packet). Progressive recovery from deadlocks is through a flitsized Deadlock Buffer central to each router which can be accessed from all neighboring nodes. System-wide, these buffers form a deadlock-free lane which can be visualized as a “floating” virtual channel. On the event of a potential deadlock situation, one of the packets in the cycle is switched to the deadlock-free lane and is routed along this path until it reaches its destination, where it is consumed to break the deadlock cycle. Potential deadlocks can be detected with a time-out mechanism. If a packet is unable to make progress for a time duration corresponding to the † Disha means “direction” in Hindi. time-out, it presumes a potential deadlock situation and becomes eligible for recovery. A packet timing out does not necessarily imply deadlock; the time-out mechanism is simply sufficient guarantee that deadlock will never occur. Disha recovers from potential deadlocks regardless of the time-out value, provided that it is finite. Network bandwidth is allocated to deadlock recovery only in the rare instances when probable deadlocks are detected; otherwise, all network bandwidth is devoted to true fully adaptive routing of normal packets. Hence, routers designed in this fashion are not only simple and potentially faster, but also can result in enhanced communication efficiency and improved network performance. Exhaustive simulations confirm that this scheme is extremely efficient even when recovery is sequentially [2]. Disha is reminiscent of Duato’s algorithm [7] and Dally and Aoki’s scheme [6] which use two virtual networks -one susceptible to deadlocks (usually fully adaptive) and the other deadlock-free (possibly deterministic). However, there are significant differences. For one, escape paths in these schemes use edge buffers. Depending on the topology (mesh or torus) more than one virtual channel might be required. However, the escape channel in Disha is a single Deadlock Buffer central to the router, a nominal resource. This buffer is shared between neighboring nodes and, unlike edge buffers, is not dedicated to any one physical path. This allows existing virtual channels to be used solely for increasing performance, not for guarding against an Destination? Begin No Yes Routing Complete !